Mixed reality graduated information delivery

ABSTRACT

Embodiments that relate to presenting a plurality of visual information density levels for a plurality of geo-located data items in a mixed reality environment are disclosed. For example, in one disclosed embodiment a graduated information delivery program receives information for a first geo-located data item and provides a first visual information density level for the item to a head-mounted display device. When a spatial information density of geo-located data item information is below a threshold, the program provides a second visual information density level greater than the first level for a second geo-located data item displayed.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/956,231, filed on Jul. 31, 2013, and titled “MIXED REALITY GRADUATEDINFORMATION DELIVERY”, the entire disclosure of which is herebyincorporated herein by reference.

BACKGROUND

Mixed reality devices such as head-mounted display devices may be usedin a variety of real-world environments and contexts. Such devices mayprovide a user with a real-time view of the physical environmentsurrounding the user, and may augment the view with virtual realityinformation, such as holographic images, text, and/or other visualinformation.

Some virtual reality information may be geo-located at a particularlocation in the physical environment. In some contexts, large amounts ofgeo-located virtual reality information may be available forpresentation to a user. With so much virtual reality informationavailable, managing the presentation of this information to a user canprove challenging. Presenting too much virtual reality information mayclutter a user's experience of a mixed reality environment and overwhelmthe user, making the information difficult to process.

In situations where a user's awareness of the immediate physicalenvironment is desirable, such as in a busy urban setting, such virtualinformation clutter may also pose a safety hazard. Additionally, in somecases the user may be interested in viewing a limited portion of thetotal amount of geo-located virtual reality information available. Inother cases, the user may desire to view more detailed informationregarding one or more items of geo-located information. Further, evenwhen a smaller portion of the available virtual reality information ispresented, the amount of information and manner of presentation maystill create a user experience that is less than desirable.

SUMMARY

Various embodiments are disclosed herein that relate to presenting aplurality of visual information density levels for a plurality ofgeo-located data items. For example, one disclosed embodiment provides,in a head-mounted display device, a method for presenting a plurality ofvisual information density levels for a plurality of geo-located dataitems in a mixed reality environment. The visual information densitylevels may comprise a minimum visual information density level and aplurality of increasing visual information density levels. The methodincludes receiving information for a selected geo-located data item. Theminimum visual information density level for the selected geo-locateddata item is provided to a display system for display by thehead-mounted display device within the mixed reality environment.

A user input corresponding to the selected geo-located data item isreceived. Based on the user input, one of the increasing visualinformation density levels for the selected geo-located data item isprovided to the display system for display by the head-mounted displaydevice within the mixed reality environment.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a mixed reality system according to anembodiment of the present disclosure.

FIG. 2 shows an example head-mounted display device according to anembodiment of the present disclosure.

FIG. 3 is a schematic illustration of a minimum visual informationdensity level and a plurality of increasing visual information densitylevels and corresponding indicators for a geo-located data item.

FIG. 4 is a schematic view of a plurality of minimum visual informationdensity level indicators corresponding to geo-located data items as seenthrough a head-mounted display device.

FIG. 5 is a schematic view of FIG. 4 in which some of the minimum visualinformation density level indicators are expanded to an increasingvisual information density level indicator.

FIG. 6 is a schematic view of a mixed reality environment including aphysical environment and a plurality of minimum visual informationdensity level indicators as viewed via a head-mounted display device.

FIG. 7 is a schematic view of a portion of the mixed reality environmentof FIG. 6 as viewed via the head-mounted display device from a locationdifferent from FIG. 6.

FIGS. 8A and 8B are a flow chart of a method for presenting a pluralityof visual information density levels for a plurality of geo-located dataitems according to an embodiment of the present disclosure.

FIG. 9 is a simplified schematic illustration of an embodiment of acomputing device.

DETAILED DESCRIPTION

FIG. 1 shows a schematic view of one embodiment of a mixed realitysystem 10. The mixed reality system 10 includes a graduated informationdelivery program 14 that may be stored in mass storage 18 of a computingdevice 22. The graduated information delivery program 14 may be loadedinto memory 26 and executed by a processor 28 of the computing device 22to perform one or more of the methods and processes described in moredetail below.

The mixed reality system 10 includes a mixed reality display program 30that may generate a virtual environment 32 for display within a physicalenvironment 34 as viewed via a display device, such as a head-mounteddisplay (HMD) device 36, to create a mixed reality environment 38. Asdescribed in more detail below, the virtual environment 32 may includegeo-located data items 40 presented in one or more visual informationdensity levels. As further described below, such geo-located data items40 may include, but are not limited to, social networking data, imagedata, recommendation/review data, and entity description data. Thegeo-located data items 40 may be presented in the one or more visualinformation density levels via images, graphics, and/or text, such asgeo-located, two-dimensional or three-dimensional images, graphics,and/or text.

The visual information density levels may include a minimum visualinformation density level 44 and a plurality of increasing visualinformation density levels, such as a first increasing visualinformation density level 48 and a second increasing visual informationdensity level 52. It will be appreciated that any suitable number ofincreasing visual information density levels may be provided. In oneexample and as described in more detail below, each increasing visualinformation density level sequentially provides a greater amount ofinformation related to a particular geo-located data item 40 thanprovided in the previous information density level.

In some examples, each such greater amount of information includes allof the information contained in the related previous information densitylevel along with additional information related to the particulargeo-located data item 40. In other examples, each such greater amount ofinformation includes none or a portion of the information contained inthe related previous information density level along with additionalinformation related to the particular geo-located data item 40.

In one example, and as described in more detail below, geo-located dataitem information 46 may be received by the computing device 22 via anetwork 54 from a server 58 or other external source. The geo-locateddata item information 46 may include a plurality of visual informationdensity levels for one or more geo-located data items 40. In otherexamples, the graduated information delivery program 14 may segregatethe geo-located data item information 46 into a plurality of visualinformation density levels. In still other examples, geo-located dataitem information 46 may be generated locally on the computing device 22.

The computing device 22 may take the form of a desktop computing device,a mobile computing device such as a smart phone, laptop, notebook ortablet computer, network computer, home entertainment computer,interactive television, gaming system, or other suitable type ofcomputing device. Additional details regarding the components andcomputing aspects of the computing device 22 are described in moredetail below with reference to FIG. 9.

The computing device 22 may be operatively connected with the HMD device36 using a wired connection, or may employ a wireless connection viaWiFi, Bluetooth, or any other suitable wireless communication protocol.Additionally, the example illustrated in FIG. 1 shows the computingdevice 22 as a separate component from the HMD device 36. It will beappreciated that in other examples the computing device 22 may beintegrated into the HMD device 36.

With reference now also to FIG. 2, one example of an HMD device 200 inthe form of a pair of wearable glasses with a transparent display 62 isprovided. It will be appreciated that in other examples, the HMD device200 may take other suitable forms in which a transparent,semi-transparent or non-transparent display is supported in front of aviewer's eye or eyes. It will also be appreciated that the HMD device 36shown in FIG. 1 may take the form of the HMD device 200, as described inmore detail below, or any other suitable HMD device. Additionally, manyother types and configurations of display devices having various formfactors may also be used within the scope of the present disclosure.Such display devices may include, but are not limited to, hand-heldsmart phones, tablet computers, and other suitable display devices.

With reference to FIGS. 1 and 2, in this example the HMD device 36includes a display system 66 and transparent display 62 that enablesgeo-located data items 40 in various visual information density levelsto be delivered to the eyes of a user 68. The transparent display 62 maybe configured to visually augment an appearance of a physicalenvironment 34 to a user 68 viewing the physical environment through thetransparent display. For example, the appearance of the physicalenvironment may be augmented by graphical content (e.g., one or morepixels each having a respective color and brightness) that is presentedvia the transparent display 62 to create a mixed reality environment 38.

The transparent display 62 may also be configured to enable a user toview a physical, real-world object in the physical environment throughone or more partially transparent pixels that are displaying a virtualobject representation. In one example, the transparent display 62 mayinclude image-producing elements located within lenses 204 (such as, forexample, a see-through Organic Light-Emitting Diode (OLED) display). Asanother example, the transparent display 62 may include a lightmodulator on an edge of the lenses 204. In this example the lenses 204may serve as a light guide for delivering light from the light modulatorto the eyes of a user. Such a light guide may enable a user to perceivea 2D image or a 3D holographic image located within the physicalenvironment that the user is viewing, while also allowing the user toview physical objects in the physical environment.

The HMD device 36 may also include various sensors and related systems.For example, the HMD device 36 may include an eye-tracking sensor system72 that utilizes at least one inward facing sensor 216. The inwardfacing sensor 216 may be an image sensor that is configured to acquireimage data in the form of eye-tracking information from a user's eyes.Provided the user has consented to the acquisition and use of thisinformation, the eye-tracking sensor system 72 may use this informationto track a position and/or movement of the user's eyes.

The HMD device 36 may also include sensor systems that receive physicalenvironment data from the physical environment 34. For example, the HMDdevice 36 may include an optical sensor system 74 that utilizes at leastone outward facing sensor 212, such as an optical sensor. Outward facingsensor 212 may detect movements within its field of view, such asgesture-based inputs or other movements performed by a user 68 or by aperson or physical object within the user's field of view. Outwardfacing sensor 212 may also capture two-dimensional image information anddepth information from physical environment 34 and physical objectswithin the environment. For example, outward facing sensor 212 mayinclude a depth camera, a visible light camera, an infrared lightcamera, and/or a position tracking camera.

The HMD device 36 may include depth sensing via one or more depthcameras. In one example, each depth camera may include left and rightcameras of a stereoscopic vision system. Time-resolved images from oneor more of these depth cameras may be registered to each other and/or toimages from another optical sensor such as a visible spectrum camera,and may be combined to yield depth-resolved video.

In other examples a structured light depth camera may be configured toproject a structured infrared illumination, and to image theillumination reflected from a scene onto which the illumination isprojected. A depth map of the scene may be constructed based on spacingsbetween adjacent features in the various regions of an imaged scene. Instill other examples, a depth camera may take the form of atime-of-flight depth camera configured to project a pulsed infraredillumination onto a scene and detect the illumination reflected from thescene. It will be appreciated that any other suitable depth camera maybe used within the scope of the present disclosure.

Outward facing sensor 212 may capture images of the physical environmentin which a user 68 is situated. In one example, the mixed realitydisplay program 30 may include a 3D modeling system that uses such inputto generate the virtual environment 32 that may model the physicalenvironment 34 surrounding the user.

The HMD device 36 may also include a position sensor system 76 thatutilizes one or more motion sensors 220 to enable position trackingand/or orientation sensing of the HMD device. For example, the positionsensor system 76 may be utilized to determine a head pose orientation ofa user's head. In one example, position sensor system 76 may comprise aninertial measurement unit configured as a six-axis or six-degree offreedom position sensor system. This example position sensor system may,for example, include three accelerometers and three gyroscopes toindicate or measure a change in location of the HMD device 36 withinthree-dimensional space along three orthogonal axes (e.g., x, y, z), anda change in an orientation of the HMD device about the three orthogonalaxes (e.g., roll, pitch, yaw).

Position sensor system 76 may also support other suitable positioningtechniques, such as GPS or other global navigation systems. Further,while specific examples of position sensor systems have been described,it will be appreciated that other suitable position sensor systems maybe used.

In some examples, motion sensors 220 may also be employed as user inputdevices, such that a user may interact with the HMD device 36 viagestures of the neck and head, or even of the body. The HMD device 36may also include a microphone system 78 that includes one or moremicrophones 224. In other examples, audio may be presented to the uservia one or more speakers 228 on the HMD device 36.

The HMD device 36 may also include a processor 230 having a logicsubsystem and a storage subsystem, as discussed in more detail belowwith respect to FIG. 9, that are in communication with the varioussensors and systems of the HMD device. In one example, the storagesubsystem may include instructions that are executable by the logicsubsystem to receive signal inputs from the sensors and forward suchinputs to computing device 22 (in unprocessed or processed form), and topresent images to a user via the transparent display 62.

It will be appreciated that the HMD device 36 and related sensors andother components described above and illustrated in FIGS. 1 and 2 areprovided by way of example. These examples are not intended to belimiting in any manner, as any other suitable sensors, components,and/or combination of sensors and components may be utilized. Thereforeit is to be understood that the HMD device 36 may include additionaland/or alternative sensors, cameras, microphones, input devices, outputdevices, etc. without departing from the scope of this disclosure.Further, the physical configuration of the HMD device 36 and its varioussensors and subcomponents may take a variety of different forms withoutdeparting from the scope of this disclosure.

With reference now to FIGS. 3-7, descriptions of example use cases andembodiments of the mixed reality system 10 will now be provided. Turningto FIG. 3 and as described in more detail below, the graduatedinformation delivery program 14 may present a geo-located data item 40in one or more visual information density levels. In the example shownin FIG. 3, the geo-located data item is a recommendation/review dataitem. The recommendation/review data item may correspond to data from arecommendation/review service that provides recommendations and/orreviews for products, services, and the like. In other examples, therecommendation/review data item may correspond to recommendation and/orreview data received from other sources, such as social networkingservices.

In the example shown in FIG. 3, the recommendation/review data item isvisually represented in a plurality of increasing visual informationdensity levels and corresponding indicator images. As described in moredetail below, a user 68 may advance from a current visual informationdensity level for a selected geo-located data item 40 to a next,increasing visual information density level via user input correspondingto the selected geo-located data item. In one example, a minimum visualinformation density level of the recommendation/review data item 304 maybe represented by an indicator, such as a star 304′. It will beappreciated that any suitable form or shape of indicator or other visualrepresentation may be utilized. As described in more detail below, anindicator corresponding to the minimum visual information density levelmay simply convey to a user that a geo-located data item 40 is locatedat the location in the physical environment 34 at which the indicator isdisplayed to the user via the HMD device 36.

Upon receiving user input directed to the star 304′ corresponding to theminimum visual information density level, a first increasing visualinformation density level of the recommendation/review data item may bedisplayed. As shown in FIG. 3, the first increasing visual informationdensity level of the recommendation/review data item may be representedby a different indicator 304″ that provides more information about thegeo-located data item 40 than the minimum visual information densitylevel indicator 304′. In the example shown in FIG. 3, the indicator 304″corresponding to the first increasing visual information density levelmay include an “R” to convey to a user that the geo-located data item isa recommendation/review data item. In this example, the first increasingvisual information density level indicator 304″ may also include adesignator “y” that corresponds to a particular recommendation/reviewservice that is providing the information.

Upon receiving user input directed to the first increasing visualinformation density level indicator 304″, a second increasing visualinformation density level of the recommendation/review data item may bedisplayed. The second increasing visual information density level of therecommendation/review data item may be represented by an indicator 304′″that is the same as the to the first information density level indicator304″, and also includes more information about the data item than thefirst information density level indicator 304″. In the example shown inFIG. 3, the second increasing visual information density level indicator304′″ includes the name of the service receiving therecommendation/review information along with a rating, such as 3 out of5 stars.

Upon receiving user input directed to the second increasing visualinformation density level indicator 304′″, a third increasing visualinformation density level indicator 304″″ corresponding to therecommendation/review data item may be displayed. The third increasingvisual information density level indicator 304″ of therecommendation/review data item provides more information about thegeo-located data item 40 than the second information density levelindicator 304′″. In the example shown in FIG. 3, the third increasingvisual information density level indicator 304″ includes an image of theAwesome Food Cart along with 3 customer review hyperlinks thatcorrespond to full text customer reviews of the food cart.

As shown in the example of FIG. 3, the increasing visual informationdensity levels may occupy increasing amounts of visual space within amixed reality environment. Advantageously and as described in moredetail below, by utilizing such increasing visual information densitylevels, the mixed reality system 10 may initially present a lowerinformation density level and correspondingly smaller indicator for aplurality of geo-located data items 40 to avoid presenting excessinformation and potentially obstructing the user's view of the physicalenvironment. Upon receiving user input selecting one or more of thegeo-located data items 40, the mixed reality system 10 may provide anincreasing visual information density level and correspondingly modifiedindicator and/or additional visual information for the selectedgeo-located data items 40.

With reference now to FIGS. 4 and 5, example schematic views ofindicators of visual information density levels displayed within aphysical environment as seen through an HMD device 200 are provided. Asshown in FIG. 4, in one example a plurality of indicators 404′, 406′,and 408′ corresponding to minimum visual information density levels aredisplayed. Each of the indicators 404′, 406′, and 408′ may correspond toone of a plurality of types of geo-located data items 40.

As shown in FIG. 4, indicators 404′, 406′ and 408′ corresponding tominimum visual information density levels occupy a relatively smallamount of visual space within the mixed reality environment 38 as viewedthrough the HMD device 200. Advantageously, in this manner the mixedreality system 10 may generally indicate to a user the location ofgeo-located data items 40 that are within the current field of view ofthe user.

Where a user desires more information regarding a particular geo-locateddata item 40 represented by one of the indictors, the user may provideuser input corresponding to the indicator and corresponding selectedgeo-located data item via the HMD device 36. In one example, the usermay gaze at indicator 406′ corresponding to a minimum visual informationdensity level for a predetermined period of time. Such predeterminedperiod of time may be, for example, 1 second, 2 seconds, 3 seconds, orany suitable period of time. With reference now to FIG. 5, using datafrom the eye-tracking sensor system 72 and upon expiration of thepredetermined period of time, the graduated information delivery program14 may replace the minimum visual information density level indicator406′ with an increasing visual information density level indicator 406″comprising additional information related to the correspondinggeo-located data item 40. In this example and as described in moredetail below, the geo-located data item is an image geo-located dataitem.

In other examples of user input, the user may provide gesture data inthe form of hand, arm, body, head, or other user movement that indicatesa geo-located data item. In other examples, user input in the form ofhead pose data comprising head orientation, position and/or location maybe used to indicate a geo-located data item. In other examples, userinput in the form of voice data may be used to indicate a geo-locateddata item. For example, a user may verbally identify a particularindicator corresponding to a geo-located data item. In other examplesand with reference again to FIG. 1, hand-held indicator data may bereceived from a hand-held indicator 86 that may be operated by the user68 to indicate a geo-located data item. For example, the position of thehand-held indicator 86 may be used by the HMD device 200 to display apointing element within the mixed reality environment 38 that the usermay guide to an indicator corresponding to a geo-located data item. Theuser 68 may then select the indictor using the hand-held indicator 86 orother form of user input.

As shown in FIG. 5, increasing visual information density levels andcorresponding indicators 404″ corresponding to a variety of differentgeo-located data items 40 may be provided. For example, arecommendation/review geo-located data item may be represented by anindicator that includes the letter “R.” An image geo-located data itemmay be represented by an indicator that includes the letter “P” and athumbnail of the image above the indicator. An image geo-located dataitem may correspond to photo, hologram, video and/or other image datafrom one or more sources.

A social networking geo-located data item may be represented by anindicator that includes the letter “S” and an icon representing theparticular social networking service above the indicator. A socialnetworking geo-located data item may correspond to data from a socialnetworking service such as, for example, posts, alerts, messages,photos, etc. An entity description geo-located data item may berepresented by an indicator that includes the letter “D.” An entitydescription geo-located data item may correspond to data related to aperson, character, company, service, location, and/or any other entity.Entity description data may include, for example, wiki entries, blogpostings, advertisements, etc. It will also be appreciated that anysuitable number and/or type of geo-located data items may be utilizedand are within the scope of the present disclosure. It will also beappreciated that the particular forms and examples of indicatorsprovided herein are for descriptive purposes, and that any othersuitable form, shape, type, of size of indicator may be utilized and arewithin the scope of the present disclosure.

In one example, 2 or more geo-located data items 40 may be located atthe same location or at locations very close to one another in the mixedreality environment 38. In this example, the 2 or more geo-located dataitems may be located at locations close enough to one another thatdisplaying a visual information density level indicator for each of the2 or more geo-located data items would cause the correspondingindicators to appear crowded to a user and/or to partially or completelyoverlap.

For example, the graduated information delivery program 14 may determinethat, as displayed via the HMD device 200, the 2 or more geo-locateddata items would be separated by a predetermined distance or less. Suchpredetermined distance may be, for example, 0.5 millimeters (mm), 1 mm,or any other suitable predetermined distance. The predetermined distancemay be measured, for example, with respect to the transparent display 62of the HMD device 200.

Based on the separation of the geo-located data items being thepredetermined distance or less, the graduated information deliveryprogram 14 may collapse the 2 or more geo-located data items into asingle, aggregated geo-located data item 80. As shown in FIG. 4, theaggregated geo-located data item 80 may be displayed as a minimum visualinformation density level indicator 408′ located at a location at ornear to the location(s) of the 2 or more geo-located data items.

In another example and with reference again to FIG. 4, an aggregatedgeo-located data item 80 may be indicated by a minimum visualinformation density level indicator 412′, and may correspond to 2 imagedata geo-located data items. A user 68 may point at the indicator 412′corresponding to the aggregated geo-located data item 80, which gestureis received via the optical sensor system 74 of the HMD device 200.

Based on the user's gesture directed to the indicator 412′, thegraduated information delivery program 14 may expand the aggregatedgeo-located data item 80 to provide an increasing visual informationdensity level for each of the two image data geo-located data items.With reference now to FIG. 5, in one example the two image datageo-located data items may be represented by indicators 506″ and 510″corresponding to increasing visual information density levels. In oneexample, each such indicator 506″ and 510″ may be linked to a locationindicator 514 representing an approximate location of the geo-locateddata item corresponding to each such indicator.

In other examples, 2 or more geo-located data items 40 located at thesame location or at locations very close to one another may berepresented by a single increasing visual information density levelindicator that indicates the number of geo-located data items at orclose to the location. For example and as shown in FIG. 5, an indicator502″ may represent 25 different geo-located data items located at orclose to a given location. In this example, the indicator includes theletter “M” to indicate that multiple geo-located data items arerepresented, along with the number 25 indicating 25 such data items.

With reference now to FIG. 6, in one example a user may be located neara city intersection including street 612 and may view a mixed realityenvironment 600 that includes the intersection via the HMD device 200.The mixed reality environment 600 may also include a plurality ofindicators 404′ and 604′ corresponding to a minimum visual informationdensity level of one or more geo-located data items 40. The minimumvisual information density level indicators 604′ are each located nearthe Awesome Food Cart 608.

The position sensor system 76 of the HMD device 200 may determinelocation data corresponding to the location of the HMD device 200. Withreference now also to FIG. 7, a user wearing the HMD device 200 may walkacross the street 612 and toward the Awesome Food Cart 608. Using thelocation data of the HMD device 200, the graduated information deliveryprogram 14 may determine that the HMD device 200 is moving closer to alocation of the minimum visual information density level indicators604′, and corresponding geo-located data items 40, near the Awesome FoodCart 608. Based on the HMD device 200 moving closer the minimum visualinformation density level indicators 604′ and corresponding geo-locateddata items 40, the graduated information delivery program 14 may provideone or more increasing visual information density levels for thecorresponding geo-located data items 40.

In the example shown in FIG. 7, the 3 corresponding geo-located dataitems 40 located above the Awesome Food Cart 608 may be represented byfirst increasing visual information density level indicators 604″. Inthis example, the first increasing visual information density levelindicators 604″ correspond to social networking geo-located data items.

In one example, the user and HMD device 200 may be closer to thegeo-located data item 40 located to the right of the Awesome Food Cart608 as compared to the geo-located data items 40 located above theAwesome Food Cart 608. Accordingly, in this example the geo-located dataitem 40 located to the right of the Awesome Food Cart 608 may berepresented by third increasing visual information density levelindicator 604″. As shown in FIG. 7, the third increasing visualinformation density level indicator 604″″ corresponds to arecommendation/review geo-located data item. In this example, the thirdincreasing visual information density level indicator 604″″ includes a 3out of 5 star rating, a photograph of the Awesome Food Cart 608, andhyperlinks 704 to 3 customer reviews of the Awesome Food Cart.

In some examples, the graduated information delivery program 14 mayprogrammatically provide one or more increasing visual informationdensity levels for a geo-located data item based on the HMD device 200moving closer to a visual information density level indicatorcorresponding to the geo-located data item, and without anycorresponding user input.

In another example, when a spatial information density in a mixedreality environment is below a predetermined threshold, instead ofproviding a minimum visual information density level for a geo-locateddata item, the graduated information delivery program 14 may provide oneof the increasing visual information density levels for the geo-locateddata item. With reference again to FIG. 7, in one example a user may beviewing the Awesome Food Cart 608 via the HMD device 200. A spatialinformation density may be defined as a percentage of a unit area of thetransparent display 62 of the HMD device 200 that is occupied bydisplayed geo-located data item information. A unit area may be, forexample, a defined region of the transparent display 62, and may bedefined by a particular area, such as 25 square millimeters (mm²), 50mm², 100 mm², or any other suitable unit area. A predetermined thresholdof the spatial information density may be, for example, 10%, 25%, 50%,75%, or any other suitable percentage of the unit area. In some examplesmultiple different unit areas and corresponding predetermined thresholdpercentages may be utilized.

With reference again to FIG. 7, in this example the geo-located dataitem 40 located to the right of the Awesome Food Cart 608 may be thesole geo-located data item located to the right of line 708 that bisectsthe transparent display 62 of the HMD device 200. Displaying the thirdincreasing visual information density level indicator 604″″ may occupy49% of the right half 712 of the transparent display 62. Where apredetermined threshold of spatial information density is 50% of onehalf of the transparent display 62, the graduated information deliveryprogram 14 may programmatically display the third increasing visualinformation density level indicator 604″″ without any user inputdirected to the corresponding geo-located data item 40.

FIGS. 8A and 8B illustrate a flow chart of a method 800 for presenting aplurality of visual information density levels for a plurality ofgeo-located data items in a mixed reality environment according to anembodiment of the present disclosure. The following description ofmethod 800 is provided with reference to the software and hardwarecomponents of the mixed reality system 10 described above and shown inFIGS. 1-7. It will be appreciated that method 800 may also be performedin other contexts using other suitable hardware and software components.

With reference to FIG. 8A, at 804 the method 800 may include receivinginformation for a selected geo-located data item. At 808 the method 800may include segregating the information into the plurality ofinformation detail levels. At 812 the method 800 may include providing aminimum visual information density level for the selected geo-locateddata item to a display system for display by a head-mounted displaydevice within the mixed reality environment.

At 816 the method 800 may include receiving a user input correspondingto the selected geo-located data item. At 820 the user input may beselected from a group consisting of eye tracking data, gesture data,head pose data, voice data, and hand-held indicator data. At 824 andbased on the user input, the method 800 may include providing anincreasing visual information density level for the selected geo-locateddata item to the display system for display by the head-mounted displaydevice within the mixed reality environment. At 830 the increasingvisual information density level may occupy an increasing amount ofvisual space within the mixed reality environment.

At 834 the method 800 may include receiving location data of thehead-mounted display device. At 838 and based on the head-mounteddisplay device moving closer to a location of the selected geo-locateddata item, the method 800 may include providing one or more of theincreasing visual information density levels for the selectedgeo-located data item to the display system for display by thehead-mounted display device. At 842 the method 800 may include, when aspatial information density in the mixed reality environment is below apredetermined threshold, instead of providing the minimum visualinformation density level for the selected geo-located data item,providing one of the increasing visual information density levels forthe selected geo-located data item.

With reference now to FIG. 8B, at 846 the method 800 may includereceiving information for two or more geo-located data items. At 850 themethod 800 may include determining that, as displayed via thehead-mounted display device, the geo-located data items would beseparated by a predetermined distance or less. At 854 and based on theseparation of the geo-located data items, the method 800 may includecollapsing the geo-located data items into an aggregated geo-locateddata item. At 858 the method 800 may include providing the minimumvisual information density level for the aggregated geo-located dataitem to the display system for display by the head-mounted displaydevice within the mixed reality environment.

At 862 the method 800 may include receiving via the head-mounted displaydevice a user input corresponding to the aggregated geo-located dataitem. At 866 and based on the user input, the method 800 may includeexpanding the aggregated geo-located data item to provide an increasingvisual information density level for each of the two or more geo-locateddata items to the display system for display by the head-mounted displaydevice within the mixed reality environment. At 870 the geo-located dataitems may be selected from a group consisting of social networking data,image data, recommendation/review data, and entity description data.

It will be appreciated that method 800 is provided by way of example andis not meant to be limiting. Therefore, it is to be understood thatmethod 800 may include additional and/or alternative steps than thoseillustrated in FIGS. 8A and 8B. Further, it is to be understood thatmethod 800 may be performed in any suitable order. Further still, it isto be understood that one or more steps may be omitted from method 800without departing from the scope of this disclosure.

FIG. 9 schematically shows a nonlimiting embodiment of a computingsystem 900 that may perform one or more of the above described methodsand processes. Computing device 22 may take the form of computing system900. Computing system 900 is shown in simplified form. It is to beunderstood that virtually any computer architecture may be used withoutdeparting from the scope of this disclosure. In different embodiments,computing system 900 may take the form of a mainframe computer, servercomputer, desktop computer, laptop computer, tablet computer, homeentertainment computer, network computing device, mobile computingdevice, mobile communication device, gaming device, etc. As noted above,in some examples the computing system 900 may be integrated into an HMDdevice.

As shown in FIG. 9, computing system 900 includes a logic subsystem 904,storage subsystem 908, display subsystem 912, and sensor subsystem 916.Computing system 900 may optionally include a communication subsystem920, an input subsystem 922 and/or other subsystems and components notshown in FIG. 9. Computing system 900 may also include computer readablemedia, with the computer readable media including computer readablestorage media and computer readable communication media. Computingsystem 900 may also optionally include other user input devices such askeyboards, mice, game controllers, and/or touch screens, for example.Further, in some embodiments the methods and processes described hereinmay be implemented as a computer application, computer service, computerAPI, computer library, and/or other computer program product in acomputing system that includes one or more computers.

Logic subsystem 904 may include one or more physical devices configuredto execute one or more instructions. For example, the logic subsystem904 may be configured to execute one or more instructions that are partof one or more applications, services, programs, routines, libraries,objects, components, data structures, or other logical constructs. Suchinstructions may be implemented to perform a task, implement a datatype, transform the state of one or more devices, or otherwise arrive ata desired result.

The logic subsystem 904 may include one or more processors that areconfigured to execute software instructions. Additionally oralternatively, the logic subsystem may include one or more hardware orfirmware logic machines configured to execute hardware or firmwareinstructions. Processors of the logic subsystem may be single core ormulticore, and the programs executed thereon may be configured forparallel or distributed processing. The logic subsystem may optionallyinclude individual components that are distributed throughout two ormore devices, which may be remotely located and/or configured forcoordinated processing. One or more aspects of the logic subsystem maybe virtualized and executed by remotely accessible networked computingdevices configured in a cloud computing configuration.

Storage subsystem 908 may include one or more physical, persistentdevices configured to hold data and/or instructions executable by thelogic subsystem 904 to implement the herein described methods andprocesses. When such methods and processes are implemented, the state ofstorage subsystem 908 may be transformed (e.g., to hold different data).

Storage subsystem 908 may include removable media and/or built-indevices. Storage subsystem 908 may include optical memory devices (e.g.,CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory devices(e.g., RAM, EPROM, EEPROM, etc.) and/or magnetic memory devices (e.g.,hard disk drive, floppy disk drive, tape drive, MRAM, etc.), amongothers. Storage subsystem 908 may include devices with one or more ofthe following characteristics: volatile, nonvolatile, dynamic, static,read/write, read-only, random access, sequential access, locationaddressable, file addressable, and content addressable.

In some embodiments, aspects of logic subsystem 904 and storagesubsystem 908 may be integrated into one or more common devices throughwhich the functionally described herein may be enacted, at least inpart. Such hardware-logic components may include field-programmable gatearrays (FPGAs), program- and application-specific integrated circuits(PASIC/ASICs), program- and application-specific standard products(PSSP/ASSPs), system-on-a-chip (SOC) systems, and complex programmablelogic devices (CPLDs), for example.

FIG. 9 also shows an aspect of the storage subsystem 908 in the form ofremovable computer readable storage media 924, which may be used tostore data and/or instructions executable to implement the methods andprocesses described herein. Removable computer-readable storage media924 may take the form of CDs, DVDs, HD-DVDs, Blu-Ray Discs, EEPROMs,and/or floppy disks, among others.

It is to be appreciated that storage subsystem 908 includes one or morephysical, persistent devices. In contrast, in some embodiments aspectsof the instructions described herein may be propagated in a transitoryfashion by a pure signal (e.g., an electromagnetic signal, an opticalsignal, etc.) that is not held by a physical device for at least afinite duration. Furthermore, data and/or other forms of informationpertaining to the present disclosure may be propagated by a pure signalvia computer-readable communication media.

Display subsystem 912 may be used to present a visual representation ofdata held by storage subsystem 908. As the above described methods andprocesses change the data held by the storage subsystem 908, and thustransform the state of the storage subsystem, the state of the displaysubsystem 912 may likewise be transformed to visually represent changesin the underlying data. The display subsystem 912 may include one ormore display devices utilizing virtually any type of technology. Suchdisplay devices may be combined with logic subsystem 904 and/or storagesubsystem 908 in a shared enclosure, or such display devices may beperipheral display devices. The display subsystem 912 may include, forexample, the display system 66 and transparent display 62 of the HMDdevice 36.

Sensor subsystem 916 may include one or more sensors configured to sensedifferent physical phenomenon (e.g., visible light, infrared light,sound, acceleration, orientation, position, etc.) as described above.Sensor subsystem 916 may be configured to provide sensor data to logicsubsystem 904, for example. As described above, such data may includeeye-tracking information, image information, audio information, ambientlighting information, depth information, position information, motioninformation, user location information, and/or any other suitable sensordata that may be used to perform the methods and processes describedabove.

When included, communication subsystem 920 may be configured tocommunicatively couple computing system 900 with one or more networksand/or one or more other computing devices.

Communication subsystem 920 may include wired and/or wirelesscommunication devices compatible with one or more differentcommunication protocols. As nonlimiting examples, the communicationsubsystem 920 may be configured for communication via a wirelesstelephone network, a wireless local area network, a wired local areanetwork, a wireless wide area network, a wired wide area network, etc.In some embodiments, the communication subsystem may allow computingsystem 900 to send and/or receive messages to and/or from other devicesvia a network such as the Internet.

When included, input subsystem 922 may comprise or interface with one ormore sensors or user-input devices such as a game controller, gestureinput detection device, voice recognizer, inertial measurement unit,keyboard, mouse, or touch screen. In some embodiments, the inputsubsystem 922 may comprise or interface with selected natural user input(NUI) componentry. Such componentry may be integrated or peripheral, andthe transduction and/or processing of input actions may be handled on-or off-board. Example NUI componentry may include a microphone forspeech and/or voice recognition; an infrared, color, stereoscopic,and/or depth camera for machine vision and/or gesture recognition; ahead tracker, eye tracker, accelerometer, and/or gyroscope for motiondetection and/or intent recognition; as well as electric-field sensingcomponentry for assessing brain activity.

The term “program” may be used to describe an aspect of the mixedreality system 10 that is implemented to perform one or more particularfunctions. In some cases, such a program may be instantiated via logicsubsystem 904 executing instructions held by storage subsystem 908. Itis to be understood that different programs may be instantiated from thesame application, service, code block, object, library, routine, API,function, etc. Likewise, the same program may be instantiated bydifferent applications, services, code blocks, objects, routines, APIs,functions, etc. The term “program” is meant to encompass individual orgroups of executable files, data files, libraries, drivers, scripts,database records, etc.

It is to be understood that the configurations and/or approachesdescribed herein are exemplary in nature, and that these specificembodiments or examples are not to be considered in a limiting sense,because numerous variations are possible. The specific routines ormethods described herein may represent one or more of any number ofprocessing strategies. As such, various acts illustrated may beperformed in the sequence illustrated, in other sequences, in parallel,or in some cases omitted. Likewise, the order of the above-describedprocesses may be changed.

The subject matter of the present disclosure includes all novel andnonobvious combinations and subcombinations of the various processes,systems and configurations, and other features, functions, acts, and/orproperties disclosed herein, as well as any and all equivalents thereof.

1. A mixed reality system for presenting a plurality of visualinformation density levels for a plurality of geo-located data items ina mixed reality environment, the mixed reality system comprising: ahead-mounted display device operatively connected to a computing device,the head-mounted display device including a display system forpresenting the plurality of visual information density levels within themixed reality environment; a memory device; and a graduated informationdelivery program stored in the memory device and executed by a processorof the computing device, the graduated information delivery programconfigured to: provide a first visual information density level for afirst geo-located data item displayed by the head-mounted display devicewithin the mixed reality environment; and when a spatial informationdensity of geo-located data item information in the mixed realityenvironment is below a predetermined threshold, provide a second visualinformation density level greater than the first visual informationdensity level for a second geo-located data item displayed by thehead-mounted display device, wherein the spatial information densitycomprises a percentage of a unit area of a display of the head-mounteddisplay device that is occupied by displayed geo-located data iteminformation.
 2. The mixed reality system of claim 1, wherein the unitarea of the display is a defined region less than an entire area of thedisplay.
 3. The mixed reality system of claim 1, further comprisingmultiple different unit areas of the display and correspondingpredetermined threshold percentages for each of the different unitareas.
 4. The mixed reality system of claim 1, wherein the graduatedinformation delivery program is configured to: receive via thehead-mounted display device a user input corresponding to the firstgeo-located data item; and based on the user input, provide anothervisual information density level greater than the first visualinformation density level for the first geo-located data item fordisplay by the head-mounted display device within the mixed realityenvironment.
 5. The mixed reality system of claim 1, wherein thegraduated information delivery program is configured to: receivelocation data of the head-mounted display device; and based on thehead-mounted display device moving closer to a location of the firstgeo-located data item, provide another visual information density levelgreater than the first visual information density level for the firstgeo-located data item to the display system for display by thehead-mounted display device.
 6. The mixed reality system of claim 1,wherein the second visual information density level occupies more visualspace within the mixed reality environment than the first visualinformation density level.
 7. The mixed reality system of claim 1,wherein the graduated information delivery program is configured to:receive information for two or more geo-located data items; determinethat, as displayed via the head-mounted display device, the two or moregeo-located data items would be separated by a predetermined distance orless; based on the separation of the two or more geo-located data items,collapse the two or more geo-located data items into an aggregatedgeo-located data item; and provide the first visual information densitylevel for the aggregated geo-located data item to the display system fordisplay by the head-mounted display device within the mixed realityenvironment.
 8. The mixed reality system of claim 7, wherein thegraduated information delivery program is further configured to: receivevia the head-mounted display device a user input corresponding to theaggregated geo-located data item; and based on the user input, expandthe aggregated geo-located data item to provide an increasing visualinformation density level for each of the two or more geo-located dataitems to the display system for display by the head-mounted displaydevice within the mixed reality environment.
 9. The mixed reality systemof claim 1, wherein the geo-located data items are selected from thegroup consisting of social networking data, image data,recommendation/review data, and entity description data.
 10. A methodfor presenting a plurality of visual information density levels for aplurality of geo-located data items in a mixed reality environment, themethod comprising: providing a first visual information density levelfor a first geo-located data item displayed by a head-mounted displaydevice within the mixed reality environment; and when a spatialinformation density of geo-located data item information in the mixedreality environment is below a predetermined threshold, providing asecond visual information density level greater than the first visualinformation density level for a second geo-located data item displayedby the head-mounted display device, wherein the spatial informationdensity comprises a percentage of a unit area of a display of thehead-mounted display device that is occupied by displayed geo-locateddata item information.
 11. The method of claim 10, wherein the unit areaof the display is a defined region less than an entire area of thedisplay.
 12. The method of claim 10, further comprising multipledifferent unit areas of the display and corresponding predeterminedthreshold percentages for each of the unit areas.
 13. The method ofclaim 10, further comprising: receiving a user input corresponding tothe first geo-located data item; and based on the user input, providinganother visual information density level greater than the first visualinformation density level for the first geo-located data item fordisplay by the head-mounted display device within the mixed realityenvironment.
 14. The method of claim 10, further comprising: receivinglocation data of the head-mounted display device; and based on thehead-mounted display device moving closer to a location of the firstgeo-located data item, providing another visual information densitylevel greater than the first visual information density level for thefirst geo-located data item to the display system for display by thehead-mounted display device.
 15. The method of claim 10, wherein thesecond visual information density level occupies more visual spacewithin the mixed reality environment than the first visual informationdensity level.
 16. The method of claim 10, further comprising: receivinginformation for two or more geo-located data items; determining that, asdisplayed via the head-mounted display device, the two or moregeo-located data items would be separated by a predetermined distance orless; based on the separation of the two or more geo-located data items,collapsing the two or more geo-located data items into an aggregatedgeo-located data item; and providing the first visual informationdensity level for the aggregated geo-located data item to the displaysystem for display by the head-mounted display device within the mixedreality environment.
 17. The method of claim 16, further comprising:receiving via the head-mounted display device a user input correspondingto the aggregated geo-located data item; and based on the user input,expanding the aggregated geo-located data item to provide an increasingvisual information density level for each of the two or more geo-locateddata items to the display system for display by the head-mounted displaydevice within the mixed reality environment.
 18. The method of claim 10,wherein the geo-located data items are selected from the groupconsisting of social networking data, image data, recommendation/reviewdata, and entity description data.
 19. In a head-mounted display device,a method for presenting a plurality of visual information density levelsfor a plurality of geo-located data items in a mixed realityenvironment, the visual information density levels comprising a minimumvisual information density level and one or more increasing visualinformation density levels, the method comprising: providing the minimumvisual information density level for a first geo-located data item to adisplay system for display by the head-mounted display device within themixed reality environment; receiving location data of a user of thehead-mounted display device; based on the user moving closer to alocation of the first geo-located data item, providing one of the one ormore increasing visual information density levels for the firstgeo-located data item to the display system for display by thehead-mounted display device as the user; and when a spatial informationdensity of geo-located data item information in the mixed realityenvironment is below a predetermined threshold, providing one of the oneor more increasing visual information density levels for a secondgeo-located data item displayed by the head-mounted display device,wherein the spatial information density comprises a percentage of a unitarea of a display of the head-mounted display device that is occupied bydisplayed geo-located data item information.
 20. The method of claim 19,wherein the increasing visual information density levels occupyincreasing amounts of visual space within the mixed reality environment.